
<!DOCTYPE html>
<html lang="">


<head><meta name="generator" content="Hexo 3.8.0">
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1.0, user-scalable=no">
  <meta name="theme-color" content="#202020">
  <meta http-equiv="x-ua-compatible" content="ie=edge">
  <script src="//busuanzi.ibruce.info/busuanzi/2.3/busuanzi.pure.mini.js" async></script>
  
  
    <meta name="keywords" content>
  

  
    <meta name="description" content="python异步编程之asyncio">
  
  
  
  <link rel="icon" type="image/x-icon" href="/images/footer-logo.png">
  
  <title>python异步编程之asyncio [ 51AIOps 专注于运维自动化  微信： kaipython ]</title>
  
    <!-- stylesheets list from config.yml -->
    
      <link rel="stylesheet" href="//cdn.bootcss.com/pure/1.0.0/pure-min.css">
    
      <link rel="stylesheet" href="/css/xoxo.css">
    
  
</head>


<body>
  <div class="nav-container">
    <nav class="home-menu pure-menu pure-menu-horizontal">
  <a class="pure-menu-heading" href="/">
    
    <span class="title" style="text-transform:none">51AIOps 专注于运维自动化  微信： kaipython</span>
  </a>

  <ul class="pure-menu-list clearfix">
      
          
            
              <li class="pure-menu-item"><a href="/" class="pure-menu-link">首页</a></li>
            
          
      
  </ul>
   
</nav>

  </div>

  <div class="container" id="content-outer">
    <div class="inner" id="content-inner" style='margin-left:-68px!important'>
      <div class="post-container">
  <article class="post" id="post">
    <header class="post-header text-center">
      <h1 class="title">
        python异步编程之asyncio
      </h1>
      <span>
        
        <time class="time" datetime="2020-01-01T16:00:00.000Z">
        2020-01-02
      </time>
        
      </span>
      <span class="slash">/</span>
      <span class="post-meta">
      <span class="post-tags">
        
      </span>
    </span>
      <span class="slash">/</span>
      <span class="read">
      <span id="busuanzi_value_page_pv"></span> 点击
    </span>
      <span class="slash">/</span>
    </header>

    <div class="post-content">
      <h3 id="asyncio-是干什么的？"><a href="#asyncio-是干什么的？" class="headerlink" title="asyncio 是干什么的？"></a>asyncio 是干什么的？</h3><ul>
<li>异步网络操作</li>
<li>并发</li>
<li>协程</li>
</ul>
<p>python3.0 时代，标准库里的异步网络模块：<code>select</code>(非常底层) python3.0时代，第三方异步网络库：<code>Tornado</code><br>python3.4时代，<code>asyncio</code>：支持 TCP ，子进程</p>
<p>现在的<code>asyncio</code>，有了很多的模块已经在支持：<code>aiohttp</code>,<code>aiodns</code>,<code>aioredis</code>等等 <a href="https://links.jianshu.com/go?to=https%3A%2F%2Fgithub.com%2Faio-libs" target="_blank" rel="noopener">https://github.com/aio-libs</a> 这里列出了已经支持的内容，并在持续更新.</p>
<p>当然到目前为止实现协程的不仅仅只有<code>asyncio</code>模块，<code>tornado</code>和<code>gevent</code>都实现了类似功能</p>
<blockquote>
<p>关于<code>asyncio</code>的一些关键字的说明：</p>
<ul>
<li><code>event_loop</code>    事件循环：程序开启一个无限循环，把一些函数注册到事件循环上，当满足事件发生的时候，调用相应的协程函数</li>
<li><code>coroutine</code> 协程：协程对象，指一个使用<code>async</code>关键字定义的函数，它的调用不会立即执行函数，而是会返回一个协程对象。协程对象需要注册到事件循环，由事件循环调用。</li>
<li><code>task</code> 任务：一个协程对象就是一个原生可以挂起的函数，任务则是对协程进一步封装，其中包含了任务的各种状态</li>
<li><code>future</code>: 代表将来执行或没有执行的任务的结果。它和<code>task</code>没有本质上的区别</li>
<li><code>async/await</code> 关键字：<code>python3.5</code>用于定义协程的关键字，<code>async</code>定义一个协程，<code>await</code>用于挂起阻塞的异步调用接口。</li>
</ul>
</blockquote>
<h3 id="简单的使用"><a href="#简单的使用" class="headerlink" title="简单的使用"></a>简单的使用</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> asyncio</span><br><span class="line"></span><br><span class="line"><span class="keyword">async</span> <span class="function"><span class="keyword">def</span> <span class="title">test</span><span class="params">(x)</span>:</span></span><br><span class="line">  print(<span class="string">'waiting...'</span>, x)</span><br><span class="line"></span><br><span class="line"><span class="keyword">async</span> <span class="function"><span class="keyword">def</span> <span class="title">main</span><span class="params">()</span>:</span>  </span><br><span class="line">  <span class="comment"># 这里是一个协程对象，这个时候test函数并没有执行</span></span><br><span class="line">  coroutine = test(<span class="number">2</span>)</span><br><span class="line">  print(coroutine)</span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> __name__ == <span class="string">'__main__'</span>:  </span><br><span class="line">  asyncio.run() <span class="comment">### 此段代码相当于下面的代码</span></span><br><span class="line">	</span><br><span class="line">  <span class="comment">#  创建一个事件loop</span></span><br><span class="line">  loop = asyncio.get_event_loop()</span><br><span class="line">  <span class="comment"># 将协程加入到事件循环loop</span></span><br><span class="line">  loop.run_until_complete(coroutine)</span><br></pre></td></tr></table></figure>

<p>在上面带中我们通过<code>async</code>关键字定义一个协程（<code>coroutine</code>），当然协程不能直接运行，需要将协程加入到事件循环<code>loop</code>中</p>
<p><code>asyncio.get_event_loop</code>：创建一个事件循环，然后使用<code>run_until_complete</code>将协程注册到事件循环，并启动事件循环。但是在3.8版本中，我们可以直接使用asyncio.run()来调用</p>
<h3 id="创建task任务"><a href="#创建task任务" class="headerlink" title="创建task任务"></a>创建task任务</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> asyncio</span><br><span class="line"></span><br><span class="line"><span class="keyword">async</span> <span class="function"><span class="keyword">def</span> <span class="title">do_some_work</span><span class="params">(x)</span>:</span></span><br><span class="line">    print(<span class="string">"waiting:"</span>,x)</span><br><span class="line">    <span class="keyword">return</span> <span class="string">"Done after &#123;&#125;s"</span>.format(x)</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">callback</span><span class="params">(future)</span>:</span></span><br><span class="line">    print(<span class="string">"callback:"</span>,future.result())</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">async</span> <span class="function"><span class="keyword">def</span> <span class="title">main</span><span class="params">()</span>:</span></span><br><span class="line">    coroutine = do_some_work(<span class="number">2</span>)</span><br><span class="line"></span><br><span class="line">    task = asyncio.create_task(coroutine)  <span class="comment">### 创建任务</span></span><br><span class="line">    print(task)</span><br><span class="line">    task.add_done_callback(callback) <span class="comment">### 回调任务结果</span></span><br><span class="line">    print(task)</span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> __name__ == <span class="string">'__main__'</span>:</span><br><span class="line">    asyncio.run(main())</span><br></pre></td></tr></table></figure>

<h3 id="并发和并行"><a href="#并发和并行" class="headerlink" title="并发和并行"></a>并发和并行</h3><p>并发指的是同时具有多个活动的系统</p>
<p>并行指的是用并发来使一个系统运行的更快。并行可以在操作系统的多个抽象层次进行运用</p>
<p>所以并发通常是指有多个任务需要同时进行，并行则是同一个时刻有多个任务执行</p>
<p>下面这个例子非常形象：</p>
<p>并发情况下是一个老师在同一时间段辅助不同的人功课。并行则是指多个老师分别同时辅助多个学生功课。简而言之就是一个人同时吃三个馒头还是三个人每人同时吃一个的情况，吃一个馒头算一个任务</p>
<h6 id="使用asyncio实现并发"><a href="#使用asyncio实现并发" class="headerlink" title="使用asyncio实现并发"></a>使用<code>asyncio</code>实现并发</h6><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> asyncio</span><br><span class="line"><span class="keyword">import</span> time</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">async</span> <span class="function"><span class="keyword">def</span> <span class="title">do_some_work</span><span class="params">(x)</span>:</span></span><br><span class="line">    now = <span class="keyword">lambda</span> :time.time()</span><br><span class="line">    print(<span class="string">'waiting...'</span>, x)</span><br><span class="line">    <span class="keyword">await</span> asyncio.sleep(x)</span><br><span class="line">    start = now()</span><br><span class="line">    print(<span class="string">"Time:"</span>,now()-start)</span><br><span class="line">    <span class="keyword">return</span> <span class="string">"Done after &#123;&#125;s"</span>.format(x)</span><br><span class="line"></span><br><span class="line"><span class="keyword">async</span> <span class="function"><span class="keyword">def</span> <span class="title">main</span><span class="params">()</span>:</span></span><br><span class="line">    coroutine1 = do_some_work(<span class="number">1</span>)</span><br><span class="line">    coroutine2 = do_some_work(<span class="number">2</span>)</span><br><span class="line">    coroutine3 = do_some_work(<span class="number">4</span>)</span><br><span class="line">    tasks = [</span><br><span class="line">        asyncio.ensure_future(coroutine1),</span><br><span class="line">        asyncio.ensure_future(coroutine2),</span><br><span class="line">        asyncio.ensure_future(coroutine3)</span><br><span class="line">    ]</span><br><span class="line">    dones, pendings = <span class="keyword">await</span> asyncio.wait(tasks)</span><br><span class="line"></span><br><span class="line">    <span class="keyword">for</span> task <span class="keyword">in</span> dones:</span><br><span class="line">        print(task.result())</span><br><span class="line"></span><br><span class="line">asyncio.run(main())</span><br></pre></td></tr></table></figure>

<p>运行结果：</p>
<figure class="highlight css"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="selector-tag">Waiting</span>: 1</span><br><span class="line"><span class="selector-tag">Waiting</span>: 2</span><br><span class="line"><span class="selector-tag">Waiting</span>: 4</span><br><span class="line"><span class="selector-tag">Task</span> <span class="selector-tag">ret</span>: <span class="selector-tag">Done</span> <span class="selector-tag">after</span> 1<span class="selector-tag">s</span></span><br><span class="line"><span class="selector-tag">Task</span> <span class="selector-tag">ret</span>: <span class="selector-tag">Done</span> <span class="selector-tag">after</span> 2<span class="selector-tag">s</span></span><br><span class="line"><span class="selector-tag">Task</span> <span class="selector-tag">ret</span>: <span class="selector-tag">Done</span> <span class="selector-tag">after</span> 4<span class="selector-tag">s</span></span><br><span class="line"><span class="selector-tag">Time</span>: 4<span class="selector-class">.004154920578003</span></span><br></pre></td></tr></table></figure>

<p>总时间为4s左右。4s的阻塞时间，足够前面两个协程执行完毕。如果是同步顺序的任务，那么至少需要7s。此时我们使用了<code>aysncio</code>实现了并发。<code>asyncio.wait(tasks)</code> 也可以使用 <code>asyncio.gather(*tasks)</code> ,前者接受一个<code>task</code>列表，后者接收一堆<code>task</code>。</p>
<h3 id="阻塞和await"><a href="#阻塞和await" class="headerlink" title="阻塞和await"></a>阻塞和await</h3><p>使用<code>async</code>可以定义协程对象，使用<code>await</code>可以针对耗时的操作进行挂起，就像生成器里的<code>yield</code>一样，函数让出控制权。协程遇到<code>await</code>，事件循环将会挂起该协程，执行别的协程，直到其他的协程也挂起或者执行完毕，再进行下一个协程的执行</p>
<p>耗时的操作一般是一些IO操作，例如网络请求，文件读取等。我们使用<code>asyncio.sleep</code>函数来模拟IO操作。协程的目的也是让这些IO操作异步化。</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> asyncio</span><br><span class="line"></span><br><span class="line"><span class="keyword">async</span> <span class="function"><span class="keyword">def</span> <span class="title">a</span><span class="params">()</span>:</span></span><br><span class="line">    print(<span class="string">'in aaa'</span>)</span><br><span class="line">    <span class="keyword">await</span> asyncio.sleep(<span class="number">2</span>)</span><br><span class="line">    print(<span class="string">'restart in a'</span>)</span><br><span class="line"></span><br><span class="line"><span class="keyword">async</span> <span class="function"><span class="keyword">def</span> <span class="title">b</span><span class="params">()</span>:</span></span><br><span class="line">    print(<span class="string">'in bbb'</span>)</span><br><span class="line">    <span class="keyword">return</span> <span class="string">'bbb'</span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">c</span><span class="params">(future)</span>:</span></span><br><span class="line">    print(future.result())</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">async</span> <span class="function"><span class="keyword">def</span> <span class="title">main</span><span class="params">()</span>:</span></span><br><span class="line">    <span class="keyword">await</span> asyncio.gather(a(), b())</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> __name__ == <span class="string">'__main__'</span>:</span><br><span class="line">    asyncio.run(main())</span><br></pre></td></tr></table></figure>


    </div>

  </article>
  <div class="toc-container">
    
  <div id="toc" class="toc-article">
    <strong class="toc-title">目录</strong>
    <ol class="toc"><li class="toc-item toc-level-3"><a class="toc-link" href="#asyncio-是干什么的？"><span class="toc-text">asyncio 是干什么的？</span></a></li><li class="toc-item toc-level-3"><a class="toc-link" href="#简单的使用"><span class="toc-text">简单的使用</span></a></li><li class="toc-item toc-level-3"><a class="toc-link" href="#创建task任务"><span class="toc-text">创建task任务</span></a></li><li class="toc-item toc-level-3"><a class="toc-link" href="#并发和并行"><span class="toc-text">并发和并行</span></a><ol class="toc-child"><li class="toc-item toc-level-6"><a class="toc-link" href="#使用asyncio实现并发"><span class="toc-text">使用asyncio实现并发</span></a></li></ol></li></ol><li class="toc-item toc-level-3"><a class="toc-link" href="#阻塞和await"><span class="toc-text">阻塞和await</span></a></li>
  </div>


  </div>
</div>
<script type="text/javascript" src="//rf.revolvermaps.com/0/0/8.js?i=5sr5du46f27&amp;m=0&amp;c=ff0000&amp;cr1=ffffff&amp;f=arial&amp;l=33" async="async"></script>
<div class="copyright">
    <span>本作品采用</span>
    <a href="https://creativecommons.org/licenses/by/4.0/">知识共享署名 4.0 国际许可协议</a>
    <span>进行许可。 转载时请注明原文链接。</span>
</div>


  
    <div class="post-nav" style="margin-left:-168px;">
      <div class="post-nav-item post-nav-next">
        
          <span>〈 </span>
          <a href="/2019/12/30/Go之channel的介绍/" rel="next" title="channel的介绍">
          channel的介绍
          </a>
        
      </div>
  
      <div class="post-nav-item post-nav-prev">
          
          <a href="/2020/01/02/LGTM,那些迷之缩写/" rel="prev" title="LGTM? 那些迷之缩写">
            LGTM? 那些迷之缩写
          </a>
          <span>〉</span>
        
      </div>
    </div>
  


	
	<div style="width:109%; margin-left:-144px" id="lv-container" data-id="city" data-uid="MTAyMC80OTg5NS8yNjM4Ng==">
	<script type="text/javascript">
   	   (function(d, s) {
       		var j, e = d.getElementsByTagName(s)[0];

       		if (typeof LivereTower === 'function') { return; }

       		j = d.createElement(s);
       		j.src = 'https://cdn-city.livere.com/js/embed.dist.js';
       		j.async = true;

       		e.parentNode.insertBefore(j, e);
   	   })(document, 'script');
	</script>
	<noscript> 为正常使用来必力评论功能请激活JavaScript</noscript>
        </div>
	



    </div>

    

  </div>
  <footer class="footer text-center">
    <div id="bottom-inner">
        <a class="bottom-item" href target="_blank">GitHub</a> |
        <a class="bottom-item" href>友情链接</a> |
        <a class="bottom-item" href="https://hexo.io" target="_blank">Powered by hexo</a> |
        <a class="bottom-item" href="https://github.com/fooying/hexo-theme-xoxo-plus" target="_blank">Theme xoxo-plus</a> |
        <a class="bottom-item" href="/atom.xml">订阅</a>
    </div>
</footer>

  

<script>
  (function(window, document, undefined) {

    var timer = null;

    function returnTop() {
      cancelAnimationFrame(timer);
      timer = requestAnimationFrame(function fn() {
        var oTop = document.body.scrollTop || document.documentElement.scrollTop;
        if (oTop > 0) {
          document.body.scrollTop = document.documentElement.scrollTop = oTop - 50;
          timer = requestAnimationFrame(fn);
        } else {
          cancelAnimationFrame(timer);
        }
      });
    }

    var hearts = [];
    window.requestAnimationFrame = (function() {
      return window.requestAnimationFrame ||
        window.webkitRequestAnimationFrame ||
        window.mozRequestAnimationFrame ||
        window.oRequestAnimationFrame ||
        window.msRequestAnimationFrame ||
        function(callback) {
          setTimeout(callback, 1000 / 60);
        }
    })();
    init();

    function init() {
      css(".heart{z-index:9999;width: 10px;height: 10px;position: fixed;background: #f00;transform: rotate(45deg);-webkit-transform: rotate(45deg);-moz-transform: rotate(45deg);}.heart:after,.heart:before{content: '';width: inherit;height: inherit;background: inherit;border-radius: 50%;-webkit-border-radius: 50%;-moz-border-radius: 50%;position: absolute;}.heart:after{top: -5px;}.heart:before{left: -5px;}");
      attachEvent();
      gameloop();
      addMenuEvent();
    }

    function gameloop() {
      for (var i = 0; i < hearts.length; i++) {
        if (hearts[i].alpha <= 0) {
          document.body.removeChild(hearts[i].el);
          hearts.splice(i, 1);
          continue;
        }
        hearts[i].y--;
        hearts[i].scale += 0.004;
        hearts[i].alpha -= 0.013;
        hearts[i].el.style.cssText = "left:" + hearts[i].x + "px;top:" + hearts[i].y + "px;opacity:" + hearts[i].alpha + ";transform:scale(" + hearts[i].scale + "," + hearts[i].scale + ") rotate(45deg);background:" + hearts[i].color;
      }
      requestAnimationFrame(gameloop);
    }

    /**
     * 给logo设置点击事件
     * 
     * - 回到顶部
     * - 出现爱心
     */
    function attachEvent() {
      var old = typeof window.onclick === "function" && window.onclick;
      var logo = document.getElementById("logo");
      if (logo) {
        logo.onclick = function(event) {
          returnTop();
          old && old();
          createHeart(event);
        }
      }
      
    }

    function createHeart(event) {
      var d = document.createElement("div");
      d.className = "heart";
      hearts.push({
        el: d,
        x: event.clientX - 5,
        y: event.clientY - 5,
        scale: 1,
        alpha: 1,
        color: randomColor()
      });
      document.body.appendChild(d);
    }

    function css(css) {
      var style = document.createElement("style");
      style.type = "text/css";
      try {
        style.appendChild(document.createTextNode(css));
      } catch (ex) {
        style.styleSheet.cssText = css;
      }
      document.getElementsByTagName('head')[0].appendChild(style);
    }

    function randomColor() {
      // return "rgb(" + (~~(Math.random() * 255)) + "," + (~~(Math.random() * 255)) + "," + (~~(Math.random() * 255)) + ")";
      return "#F44336";
    }

    function addMenuEvent() {
      var menu = document.getElementById('menu-main-post');
      if (menu) {
        var toc = document.getElementById('toc');
        if (toc) {
          menu.onclick = function() {
            if (toc) {
              if (toc.style.display == 'block') {
                toc.style.display = 'none';
              } else {
                toc.style.display = 'block';
              }
            }
          };
        } else {
          menu.style.display = 'none';
        }
      }
    }

  })(window, document);
</script>

  



  

</body>
</html>
